Foundation Models for Natural Language Processing

Connectionist Models for Natural Language Processing Program

The scientific adequacy of models based on a small number of coarse-grained primitives (e.g. conceptual dependency), popular in AI during the 70's, has been called into question and substantially replaced by a current emphasis in much of computational linguistics on lexicalist models (i.e., ones which use words for representing concepts or meanings). However, few people can doubt that words are...

Maximum Entropy Models for Natural Language Processing

Natural Language Processing as a Foundation of the Semantic Web

The main argument of this paper is that Natural Language Processing (NLP) does, and will continue to, underlie the Semantic Web (SW), including its initial construction from unstructured sources like the World Wide Web (WWW), whether its advocates realise this or not. Chiefly, we argue, such NLP activity is the only way up to a defensible notion of meaning at conceptual levels (in the original ...

Unsupervised Natural Language Processing Using Graph Models

In the past, NLP has always been based on the explicit or implicit use of linguistic knowledge. In classical computer linguistic applications explicit rule based approaches prevail, while machine learning algorithms use implicit knowledge for generating linguistic knowledge. The question behind this work is: how far can we go in NLP without assuming explicit or implicit linguistic knowledge? Ho...

A Primer on Neural Network Models for Natural Language Processing

Over the past few years, neural networks have re-emerged as powerful machine-learning models, yielding state-of-the-art results in fields such as image recognition and speech processing. More recently, neural network models started to be applied also to textual natural language signals, again with very promising results. This tutorial surveys neural network models from the perspective of natura...